1. Field of the Invention
The present invention relates to a system, method, and program for detecting approach to an object based on an optical flow calculated by image processing.
2. Description of the Related Art
In order to avoid a collision of a movable agent against an object, a method using an algorithm is known (refer to David N Lee, xe2x80x9cA Theory of Visual Control of Braking Based on Information about Time-to-Collisionxe2x80x9d, Perception, Vol. 5, pp. 437-459, 1976). In this algorithm, a time from the current time to an estimated collision with the object is calculated based on optical flow data. Typically, a camera is attached to the movable agent and image processing is performed using this algorithm based on images taken by the camera, thereby calculating the time at which the movable agent will just reach the object. That is, the algorithm can be effectively used for detecting approach of a movable agent to an object.
However, this conventional method does not consider the pitch and roll of the movable agent itself during the approach of the movable agent; thus, it may be difficult to detect an accurate degree of the approach. Here, when the movable agent moves, the motion includes swinging or vibration. Therefore, each optical flow calculated based on the images in the actual situation, which are taken by a camera attached to the movable agent, has an approach component (i.e., a component in the direction in which the movable agent approaches to the object) which includes a vibration component in the yaw (i.e., horizontal) direction, the pitch (i.e., vertical) direction, or the like. In the detection of approach to an obstacle based on the optical flow data, these vibration components function as noise components which cause erroneous detection. The applicant of the present invention proposed a method for detecting approach to an obstacle in Japanese Unexamined Patent Application, First Publication No. Hei 11-134504; however, a detection or recognition method having a higher accuracy has been required.
In consideration of the above circumstances, an object of the present invention is to provide a system, method, and program for accurately detecting approach to an object by performing simple processing even if processed images include a vertical or horizontal vibration component.
Therefore, the present invention provides a system for detecting approach to an object based on time-series images of a surrounding environment, comprising:
an optical flow calculating section for calculating an optical flow based on at least two time-series images;
a first adding section for calculating an added value for detecting approach, by adding values corresponding to data of the optical flow assigned to a first predetermined area defined on the images, where a predetermined filter function is used in the adding calculation;
a second adding section for calculating an added value for detecting vibration, by adding values corresponding to data of the optical flow assigned to a second predetermined area defined on the images, where said filter function is used in the adding calculation; and,
a situation determining section for determining that approach to the object has occurred when a difference between the added values for detecting approach and vibration exceeds a predetermined threshold value.
According to this structure, even when the image data include a vibration component in a vertical or horizontal direction, the approach to the object can be detected with high accuracy by simply calculating a difference between the added values (each assigned to a specific predetermined area) for detecting approach and vibration of the optical flow.
As a typical example, the second predetermined area is defined in a manner such that the second predetermined area does not include a source of the optical flow on the images; and the first predetermined area is defined further from the source of the optical flow in comparison with the second predetermined area. In this case, when the approach to the object proceeds, the difference between the added value related to a component for detecting approach and the added value related to a yaw or pitch component, which acts as a noise component, can be large, thereby improving the detecting accuracy and easily determining and detecting a collision.
The vibration component included in the image has large noise components in the yaw and pitch directions. In order to efficiently remove the noise components, the first and second predetermined areas may be defined so as to calculate the added values for detecting approach and vibration by using at least one of a yaw component and a pitch component of the optical flow.
The situation determining section may include:
a noise removing section for removing a high-frequency noise component from each of the added values for detecting approach and vibration by using a temporal filter based on a predetermined time constant;
a weighting section for multiplying each of the added values for detecting approach and vibration, which include no high-frequency noise component, by a predetermined weight coefficient;
a subtracting section for calculating a difference between the weighted added value for detecting approach and the weighted added value for detecting vibration; and
a determining section for determining whether the difference between the weighted added values for detecting approach and vibration exceeds a predetermined threshold value.
According to the above structure, which has the section for removing a high-frequency noise component from each of the two added values, and the section for weighting the two added values, the added values can be suitably corrected according to the moving speed of a movable agent, and the angle of visibility, the resolution, and the sampling frequency of the image, thereby improving the detecting accuracy.
The present invention also provides a method of detecting approach to an object based on time-series images of a surrounding environment, comprising:
an optical flow calculating step for calculating an optical flow based on at least two time-series images;
a first adding step for calculating an added value for detecting approach, by adding values corresponding to data of the optical flow assigned to a first predetermined area defined on the images, where a predetermined filter function is used in the adding calculation;
a second adding step for calculating an added value for detecting vibration, by adding values corresponding to data of the optical flow assigned to a second predetermined area defined on the images, where said filter function is used in the adding calculation; and,
a situation determining step for determining that approach to the object has occurred when a difference between the added values for detecting approach and vibration exceeds a predetermined threshold value.
According to this method, even when the image data include a vibration component in a vertical or horizontal direction, the approach to the object can be detected with high accuracy by simply calculating a difference between the added values (each assigned to a specific predetermined area) for detecting approach and vibration of the optical flow.
The situation determining step may include:
a noise removing step for removing a high-frequency noise component from each of the added values for detecting approach and vibration by using a temporal filter based on a predetermined time constant;
a weighting step for multiplying each of the added values for detecting approach and vibration, which include no high-frequency noise component, by a predetermined weight coefficient;
a subtracting step for calculating a difference between the weighted added value for detecting approach and the weighted added value for detecting vibration; and
a determining step for determining whether the difference between the weighted added values for detecting approach and vibration exceeds a predetermined threshold value.
According to the above method, which has the step of removing a high-frequency noise component from each of the two added values, and the step of weighting the two added values, the added values can be suitably corrected according to the moving speed of a movable agent, and the angle of visibility, the resolution, and the sampling frequency of the image, thereby improving the detecting accuracy.
The present invention also provide a program for making a computer execute an operation for detecting approach to an object based on time-series images of a surrounding environment, the operation comprising:
an optical flow calculating step for calculating an optical flow based on at least two time-series images;
a first adding step for calculating an added value for detecting approach, by adding values corresponding to data of the optical flow assigned to a first predetermined area defined on the images, where a predetermined filter function is used in the adding calculation;
a second adding step for calculating an added value for detecting vibration, by adding values corresponding to data of the optical flow assigned to a second predetermined area defined on the images, where said filter function is used in the adding calculation; and,
a situation determining step for determining that approach to the object has occurred when a difference between the added values for detecting approach and vibration exceeds a predetermined threshold value.